Feature extraction method, object classification method, object identification method, feature extraction device, object classification device, object identification device, feature extraction/object classification/object identification program, and recording medium on which the program is recorded

ABSTRACT

Provided is a feature extraction that extracts a feature that represents a characteristic of a subject, the feature being extracted from an image that has imaged the subject, the feature being extracted without relation to the shape of the subject. The feature extraction extracts the feature from the image of the subject, the subject having been imaged by an imaging means.

TECHNICAL FIELD

The present invention relates to a feature extraction method, objectclassification method, object identification method, feature extractiondevice, object classification device, object identification device,feature extraction/object classification/object identification program,and recording medium on which the program is recorded, and inparticular, relates to a method for extracting the feature descriptionof a soft item, irrespective of the shape of the soft item, from animage obtained by imaging the soft item as a subject, an objectclassification method and an object identification method in which theextracted feature description is used, an object classification deviceand an object identification device, and a program for causing acomputer to function as a feature extraction device, objectclassification device, and object identification device.

BACKGROUND ART

Today, mechanization has progressed in a variety of fields due totechnological advancements. When an object is operated using a robot,there is a need to identify the position, type, and other attributes ofthe object; therefore, an image obtained by a CCD camera or other meansis processed and analyzed, thereby fulfilling a function correspondingto the human eye.

A variety of types of objects exist in the everyday environment of humanlifestyle, from rigid bodies that have shapes that do not change at allto objects such as cloth or paper that change to a variety of shapes.When performing automation using a robot, there is a need to identifywhat the object is from an image irrespective of the shape of theobject.

In recent years, SIFT (non-patent document 1) is commonly used foridentification of objects by image processing. This method enables imageprocessing even when the manner in which the object is visible changesto a certain degree, but is image processing that basically assumes thatthe object is a rigid body, and is therefore difficult to apply foridentification of, e.g., (1) soft items such as clothing that may assumea variety of shapes through folded overlapping, bending, creasing, orthe like, (2) asphalt portions of roads and bare earth or grass portionson the shoulder, (3) foreign objects that are mixed in or layered ontothe subject, such as dust on a floor, or (4) vegetables and fruits,which assume different outer shapes even between the same type due to,e.g., bending or the shape of leaves. Therefore, in order to providelifestyle assistance in the everyday environment of humans, there is aneed for, e.g., (1) an image processing method for appropriatelyidentifying soft items, in light of cases in which automated machinessuch as robots handle laundry, which is a soft item; (2) an imageprocessing method for an automated travel system in which electricwheelchairs or automobiles used by the visually impaired travel alongdesignated positions; and (3) an image processing method for identifyingsoiled portions on the floor when an automatic vacuum cleaner cleans thefloor. In addition, there is also a need for (4) an image processingmethod for accurately classifying and/or identifying objects that mayassume a variety of outer shapes in industry settings such asclassification of vegetables and fruits in a food factory.

With regards to image processing methods in relation to soft items, avariety of image features have been conventionally used. For example,Kakikura et al. realized an isolation task using color information (seenon-patent document 2). Ono et al. proposed a method for expressing,with regards to a square cloth product such as a handkerchief, a statein which a part of the product is folded (see non-patent document 3).Kita et al. proposed a method for using a three-dimensional variableshape model, and applying the model to a group of three-dimensionalpoints obtained by measurement (see non-patent document 4). However, inthese existing studies, the type of cloth product is provided inadvance, or identification information for specifying the cloth productis defined as, e.g., the color of the material, and information forspecifying the cloth product or the like is necessary in advance.

If there is a method making it possible to extract feature descriptionsof a cloth product or the like from an image that can be genericallyused without requiring information for specifying the product asdescribed above, it is possible to classify a plurality of types ofproducts into identical products for lifestyle support or in a cleaningplant. In addition, if there is a method making it possible to identifythe classified product, the method might be useful for automation usinga robot or the like. Accordingly, Osawa et al. (see non-patent document5), and Abbeel et al. (see

non-patent document 6) propose methods for identifying the outline orthe position of the bottom end point of a cloth product while a robot isoperating the cloth product, and identifying the type of the clothproduct.

PRIOR ART DOCUMENTS Non-Patent Documents

Non-patent document 1: D. G. Lowe “Distinctive image features fromscale-invariant keypoints,” Int'l Journal of Computer Vision, vol. 60,No. 2, pp. 91-110, 2004.

Non-patent document 2: K. Hamajima and M. Kakikura: “Planning Strategyfor Unfolding Task of Clothes—Isolation of clothes from washed mass—,”in Proc. of Int'l Conf. on Robots and Systems, pp. 1237-1242, 2000

Non-patent document 3: E. Ono, H. Okabe, H. Ichijo and N. Aisaka “RobotHand with Sensor for Cloth Handling,” In Proc. 1990, Japan, USA Symp. onFlexible Automation, pp. 1363-1366, 1990

Non-patent document 4: Y. Kita, F. Saito and N. Kita: “A deformablemodel driven method for handling clothes,” Proc. of Int. Conf. onPattern Recognition, 2004.

Non-patent document 5: F. Osawa, H. Seki, and Y. Kamiya, “Unfolding ofMassive Laundry and Classification Types by Dual Manipulator,” Journalof Advanced Computational Intelligence and Intelligent Informatics, Vol.11 No. 5, pp. 457-, 2007.

Non-patent document 6: J. Maitin-Shepard, M. Cusumano-Towner, J. Lei andP. Abbeel: “Cloth Grasp Point Detection based on Multiple-View GeometricCues with Application to Robotic Towel Folding,” Int'l. Conf. onRobotics and Automation, pp. 2308-2315, 2010.

Non-patent document 7: Y. Zhao and G. Karypis: “Comparison ofagglomerative and partitional document clustering algorithms,”University of Minnesota-Computer Science and Engineering TechnicalReport, No. 02-014, 2002 Non-patent document 8: C. Chang and C. Lin:“LIVSVM”.

DISCLOSURE OF INVENTION Problems to be Solved by Invention

However, in each of the aforementioned image processing methods, a softitem is subjected to a physical operation, and it is difficult toidentify the type and the like of soft items in a state of beingfoldedly overlapped or a state of being crumpled into an irregularshape, such as a handkerchief that has been taken out of a pocket orclothing that has been taken out from a clothes basket. In addition, itis also difficult to identify objects that are mixed in or layered on asubject, or to classify or identify objects that have differentappearances even between the same types.

As a result of intense research, the inventors of the present inventionhave newly discovered a method for extracting a feature descriptionrepresenting a consistent feature irrespective of the outer shape suchas creasing or folded overlapping as long as the material is identicalor irrespective of any difference in outer shape due to individualdifferences as long as the type of object is identical, and a method forextracting a feature description representing a consistent feature foreach object irrespective of the shape of different objects mixed in orlayered on a subject. The inventors also discovered that classifyingsubjects on the basis of the feature description extracted by thefeature extraction method makes it possible to classify the subjectsinto the same classification, and that making a comparison with thefeature description of a subject that is already known makes it possibleto identify what the subject is, and arrived at the present invention.

Specifically, an object of the present invention is to provide a methodfor extracting, from an image obtained by imaging a subject, a featuredescription representing a feature of the subject irrespective of theshape of the subject, and a method for classifying and a method foridentifying a subject using the extracted feature description. Anotherobject of the present invention is to provide an object classificationdevice and an object identification device in which the featureextraction method is used. Another object of the present invention is toprovide a program for causing a computer to function as a featureextraction device, an object classification device, or an objectidentification device, and a recording medium on which the program isrecorded.

Means for Solving the Problem

The present invention is the following feature extraction method, objectclassification method, object identification method, feature extractiondevice, object classification device, object identification device,feature extraction/object classification/object identification program,and recording medium on which the program is recorded.

(1) A feature extraction method for extracting a feature descriptionfrom an image of a subject captured by image-capturing means, thefeature extraction method characterized in having:

a step for creating a filter bank from the image;

a step for creating a maximum brightness image from the filter bank;

a step for setting a circular image region of the maximum brightnessimage, and setting a center Cc and a radius Rc of the circular imageregion;

a step for projecting pixels in the maximum brightness image in athree-dimensional space having axes representing (a) the ratio betweenthe distance L_(D) between the pixel position (x, y) and the center Cc,and the radius Rc, (b) the brightness value F_(D)(x, y) of the pixel,and (c) the total of the difference between the brightness value F_(D)of the pixel and the brightness value of a nearby pixel; and a step forcreating a frequency histogram from the pixels projected in thethree-dimensional space.

(2) A feature extraction method for extracting a feature descriptionfrom an image of a subject captured by image-capturing means, thefeature extraction method characterized in having:

a step for creating a filter bank from the image;

a step for creating a maximum brightness image from the filter bank;

a step for setting a circular image region of the maximum brightnessimage, and setting a center Cc and a radius Rc of the circular imageregion;

a step for projecting pixels in the maximum brightness image in athree-dimensional space having axes representing (d) the ratio betweenthe distance L_(O) between the position (x, y) of a pixel of interestand the center Cc, and the radius Rc, (e) a value E_(O) in which whetherthe pixel of interest is positioned on the upper side or the lower sideof a folded overlap is evaluated by a continuous value, and (f) adirection component of the folded overlap portion in which the pixel ofinterest is present; and

a step for creating a frequency histogram from the pixels projected inthe three-dimensional space.

(3) The feature extraction method according to (1) or (2), characterizedin the subject being a soft item.

(4) The feature extraction method according to any of (1) through (3),characterized in the filter bank being created using a Gabor filter.

(5) An object classification method, characterized in a subject beingclassified using a frequency histogram extracted using the featureextraction method according to any of (1) through (4).

(6) An object identification method, characterized in a frequencyhistogram extracted using the feature extraction method according to anyof (1) through (4) being compared to a frequency histogram of a knownsubject.

(7) The object identification method according to (6), characterized ina plurality of types of the known subject existing, and the extractedfrequency histogram being compared with a frequency histogram of theplurality of known-subject types, whereby the type of the subject isidentified.

(8) A feature extraction device for extracting a feature descriptionfrom an image of a subject captured by image-capturing means, thefeature extraction device characterized in having:

filter bank creation means for creating a filter bank from the image;

filtering result synthesizing means for creating a maximum brightnessimage from the filter bank;

maximum brightness image center and radius setting means for setting acircular image region of the maximum brightness image, and setting acenter Cc and a radius Rc of the circular image region;

three-dimensional projection means for projecting pixels in the maximumbrightness image in a three-dimensional space having axes representing(a) the ratio between the distance L_(D) between the pixel position (x,y) and the center Cc, and the radius Rc, (b) the brightness valueF_(D)(x, y) of the pixel, and (c) the total of the difference betweenthe brightness value F_(D) of the pixel and the brightness value of anearby pixel; and frequency histogram creation means for creating afrequency histogram from the pixels projected in the three-dimensionalspace.

(9) A feature extraction device for extracting a feature descriptionfrom an image of a subject captured by image-capturing means, thefeature extraction device characterized in having:

filter bank creation means for creating a filter bank from the image;

filtering result synthesizing means for creating a maximum brightnessimage from the filter bank;

maximum brightness image center and radius setting means for setting acircular image region of the maximum brightness image, and setting acenter Cc and a radius Rc of the circular image region;

three-dimensional projection means for projecting pixels in the maximumbrightness image in a three-dimensional space having axes representing(d) the ratio between the distance L_(O) between the position (x, y) ofa pixel of interest and the center Cc, and the radius Rc, (e) a valueE_(O) in which whether the pixel of interest is positioned on the upperside or the lower side of a folded overlap is evaluated by a continuousvalue, and (f) a direction component of the folded overlap portion inwhich the pixel of interest is present; and frequency histogram creationmeans for creating a frequency histogram from the pixels projected inthe three-dimensional space.

(10) The feature extraction device according to (8) or (9),characterized in the subject being a soft item.

(11) The feature extraction device according to any of (8) through (10),characterized in the filter bank being created using a Gabor filter.

(12) An object classification device, characterized in the objectclassification device having image classification means for classifyinga subject using a frequency histogram extracted using the featureextraction device according to any of (8) through (11).

(13) An object identification device, characterized in the objectidentification device having: an identification database storing afrequency histogram of a known subject, and identification means forcomparing, with a frequency histogram of a known subject stored in theidentification database, and identifying a frequency histogram extractedby the feature extraction device according to any of (8) through (12).

(14) The object identification device according to claim 13,characterized in that:

the frequency histograms of the known subject stored in theidentification database are frequency histograms of a plurality of typesof subjects, and

the identification means compares the extracted frequency histogram withthe plurality of types of frequency histograms stored in theidentification database, and thereby identifies the type of the subject.

(15) A program, characterized in causing a computer to function as thefeature extraction device according to one of (8) through (11), theobject classification device according to (12), or the objectidentification device according to (13) or (14).

(16) A computer-readable recording medium in which the program disclosedin (15) is recorded.

Effect of the Invention

In the present invention, it is possible to extract a featuredescription that does not depend on the outer shape of a subject, evenif there is a difference in the outer shape of the subject due tocreasing, folded overlapping, or the like, a difference in the outershape due to individual differences between objects, or a difference inthe outer shape of an object mixed in or layered on a subject, thereforemaking it possible to classify and identify the subject keeping anintact shape without using physical means. In addition, theimage-capturing means does not have to image the subject to a highdefinition; an image having a relatively low resolution will suffice.Performing a comparison with the feature description of a known subjectmakes it possible to identify the type of the subject, irrespective ofthe shape of the subject. In addition, in the feature extraction method,the object classification method, and the object identification methodof the present invention, there is no comparison made between the outershape or the size of the subject. Instead, the appearance of an existingsurface due to {features such as} creases and folded overlaps caused bythe material or type of the subject, patterns of or on the leaf bladesor leaf veins, the presence and density of stalks, stems, or fruits, theearth/sand type, or the type or amount of deposited or adhered matter,or the roughness or the like of the material or the surface finishing ofa hard substance, is extracted as a feature description for performingclassification and identification. Therefore, the subject can beclassified and identified by processing at least one image imaged froman arbitrary angle. As a result, the amount of data required for theclassification and identification is minimized, and the processing speedof the required device can thereby be improved.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 shows an overview of a (1) object identification device, (2)object classification device, and (3) feature extraction device of thepresent invention;

FIG. 2 is a block diagram showing the detailed configuration of (1) afeature extraction unit and image identification unit of the objectidentification device and (2) a feature extraction unit and imageclassification means of the object classification device of the presentinvention;

FIG. 3 shows the steps of a feature extraction method of the presentinvention;

FIG. 4 shows the specific steps of filter bank creation;

FIG. 5 shows the configuration of edge detection filters having varyingdirections and sizes, the shape of the filter being defined by theamplitude and window width;

FIG. 6 shows the center Cc and radius Rc of a circular clothing regionset from the maximum brightness image;

FIG. 7 shows an overview of the CM-WD method (feature extraction methodfocusing on the density of creases and the cloth material) of thepresent invention, and shows a designated pixel and surrounding pixels,with the center Cc of the maximum brightness image region being theorigin;

FIG. 8 shows an overview of pixel number count in each three-dimensionalgrid obtained by projecting feature descriptions in a three-dimensionalspace and partitioning the three-dimensional space into n³three-dimensional grids;

FIG. 9 shows a frequency histogram, the horizontal axis representinggrids partitioned into n³ grids, and the vertical axis representing thenumber of pixels present in each grid;

FIG. 10 shows an overview of the OVLP method (feature extraction methodfocusing on the position and direction of folded overlaps on the cloth)according to the present invention, and shows dividing pixelssurrounding the pixel of interest being divided into the upper side andthe lower side of the folded overlap, with the center Cc of the targetregion being the origin, with respect to each pixel on the edgeextracted as the folded overlap portion;

FIG. 11 shows an overview of the DST-W method (feature descriptionregarding the distribution and direction of creases) according to acomparative example, and shows the relative angle between a line linkingCc to the center coordinates of the long axis of an elliptical regionand the long axis of the elliptical region;

FIG. 12 shows an example of the steps for creating an identificationdatabase;

FIG. 13 is a photograph-substituted drawing, in which photographs showcloth products used in the embodiment;

FIG. 14 is a graph showing the identification rates obtained whenidentification databases were created using feature descriptionsextracted according to the feature extraction methods of the presentinvention and the feature descriptions extracted using the methodsaccording to the comparative examples and cloth products wereidentified; and

FIG. 15 is a graph showing the identification rates when identificationdatabases were created through combing feature descriptions.

DESCRIPTION OF EMBODIMENTS

A best mode for carrying out the present invention will now be describedusing a drawing.

A feature extraction method, object classification method, and objectidentification method according to the present invention are carriedout, e.g., using a device such as that shown in FIG. 1.

Specifically, FIG. 1(1) shows an example of an object identificationdevice 100, comprising: image-capturing means 110 for imaging a subject;a feature extraction unit 120 for extracting a feature description fromthe imaged image; an image identification unit 130 having the outputfrom the feature extraction unit 120 supplied thereto and performingidentification of the subject; a control unit 140 for controlling theimage-capturing means 110, the feature extraction unit 120, and theimage identification unit 130; and a program memory 150, which isaccessed from the control unit 140. The subject may be imaged by theimage-capturing means 110 by a command from the control unit 140 whenthe subject is placed in the object identification device and thepresence of the subject is confirmed by a sensor (not shown), or placedin front of the image-capturing means 110 without using a sensor or thelike and imaged by manual operation. In an instance in which the featuredescription is extracted from an image of the subject imaged using anexternal image-capturing means and identification of the subject isperformed, an arrangement obtained by omitting the image-capturing means110 in FIG. 1(1) may be used as the object identification device 100.

In an instance in which the subject is classified without theidentification of the subject being performed, providing imageclassification means 230 instead of the image identification unit 130 asshown in FIG. 1(2) will result in an object classification device 200being obtained. In an instance in which only the extraction of thefeature description from the image is performed without the subjectbeing classified or identified, a feature extraction device 300 can beobtained from the image-capturing means 110, feature extraction unit120, control unit 140, and program memory 150 as shown in FIG. 1(3).

In an instance in which the feature description is extracted from animage of the subject captured by external image-capturing means in theobject classification device 200 or the feature extraction device 300,the image-capturing means 110 can be omitted. In FIGS. 1(1) through1(3), elements being affixed with identical numerals indicate that theelements have an identical function.

In the present invention, a feature description refers to informationfor classifying or identifying a subject from an image of the subjectimaged by the image-capturing means. A feature description of thepresent invention is extracted by creating a maximum brightness imagefrom the image of a subject imaged by the image-capturing means,projecting the pixels of the maximum brightness image in athree-dimensional space, and creating a frequency histogram from thepixels, as disclosed in claims 1 and 2. Specifically, {a featuredescription} signifies information representing the external appearanceof the subject, such as: shape information such as bends, creases,folded overlaps, patterns of or on leaf blades or leaf veins, or thepresence and density of stalks, stems, or fruits; information regardingthe material constituting the subject itself such as the earth/sandtype, or the roughness of the material or the surface finishing of ahard substance; or information regarding the type or amount of depositedmatter deposited on an existing surface or adhered matter adhering tothe existing surface, obtained by computer-processing an image. Thefeature description in the present invention may be a featuredescription extracted from a subject comprising a single object, orindividual feature descriptions of different objects mixed in or layeredon a subject comprising a plurality of objects. If a plurality ofobjects are present in the image, it is possible to perform a process ofdividing the image into regions that appear different in the image, useeach divided region as a subject, and extract a feature description fromeach region such as “road” or “plant bed.” No particular limitationsexist with regards to the subject to which the present invention can beapplied; the subject may be one that does not readily change shape toone that does.

Examples of the material constituting the soft item include: cotton,silk, wool, cellulose; natural materials such as regenerated material,such as rayon, polynosic, and cupra, obtained using cellulose includedin natural wooden material or the like; semi-synthetic natural materialssuch as acetate, triacetate, and promix synthesized from naturalcellulose, an animal protein, or the like and a chemical material suchas acetic acid; synthetic chemical materials obtained from petroleum orthe like, such as polyamide, acrylic, nylon, aramid, vinylon,polypropylene, polyethylene, polyurethane, polyvinyl chloride,vinylidene, and polyester; organic materials such as carbon, rubber, andplant materials such as wood; inorganic materials such as silicon andglass; and metal materials such as iron, steel, and stainless steel.Products are obtained from, e.g., flakes, yarns, fibers, and wires, aswell as knitted fabrics, textiles, lattices, felts obtained byintertwining fibers or the like, and unwoven cloths, in which an abovematerial is used. Specific examples include soft items which changeshape by creasing, folded overlapping, or the like, such as cloths,items of clothing, papers, and metal meshes.

Examples of objects that have different outer shapes due to individualdifferences despite individual objects being relatively rigid includeplants, animals, food items (breads, noodles), and the like. Examples ofthe plants include vegetables such as cucumber, daikon, carrot,eggplant, tomato, spinach, Chinese cabbage, and cabbage, and fruits suchas peach, apple, tangerine, strawberry, and grape. Examples of theanimals include mammals, birds, fishes, reptiles, and insects. Forexample, by capturing expanded images of body surface portions of dogsand recording the captured images as, e.g., “Doberman coat,” it ispossible to identify dog breeds on the basis of the coat. Birds, fishes,reptiles, insects, and the like can be similarly identified.

Examples of individual objects present in a subject include asphalt aswell as grass or earth on the road shoulder in a photograph of a road,and buildings in an aerial photograph.

Examples of foreign objects layered on an object in the subject includedust, paper scraps, sauce and other stains, animal hair, and food scrapson a floor. In order to identify whether a plurality of objects arelayered, or not layered, in the image, it is necessary to define, inadvance, the state in which the objects are layered and the state inwhich the objects are not layered as different states.

In the program memory 150, there are stored, in advance, e.g., a programfor causing the computer shown in FIG. 1(1) to function as an objectidentification device, a program for causing the computer shown in FIG.1(2) to function as an object classification device, and a program forcausing the computer shown in FIG. 1(3) to function as a featureextraction device. The program is read and executed by the control unit140, whereby actuation of an image-capturing means 110, a featureextraction unit 120, an image identification unit 130, or imageclassification means 230 described further below is controlled. Theprograms may alternatively be recorded in a recording medium and storedin the program memory 150 using installation means.

FIG. 2(1) illustrates the detailed configuration of the featureextraction unit 120, the image identification unit 130, or the imageclassification means 230. The feature extraction unit 120 includes atleast: filter bank creation means 121 for creating a filter bank fromthe imaged image; filtering result synthesizing means 122 forsynthesizing a filter image obtained by the filter bank creation means121 and creating a maximum brightness image; maximum brightness imagecenter and radius setting means 123 for setting a circular image regionof the maximum brightness image obtained by the filtering resultsynthesis means 122 and setting the center Cc and the radius Rc of thecircular image region; three-dimensional projection means 124 forprojecting the maximum brightness image in a three-dimensional space;and frequency histogram creation means 125 for creating a frequencyhistogram from pixels projected by the three-dimensional projectionmeans 124. No particular limitations exist with regards to theimage-capturing means 110, as long as the subject can be imaged as adigital data image. In addition, there is no need to image a clothmaterial to a high definition; a relatively low-resolution image isacceptable. In a verification experiment, it was possible to extract afeature description enabling identification of the subject, even whenthe number of pixels is 640 horizontally and 480 vertically, i.e.,approximately 300,000. This resolution is equal to or less than onetenth of that of a commercially available digital camera. No particularlimitations exist with regards to the necessary number of pixels as longas the number is within a range in which the feature description can beextracted, even if the number of pixels is less than 300,000. Havingmore than 300,000 pixels will not present problems in terms of featureextraction, but an excessively large pixel number will reduce the speedof feature extraction. Therefore, the number of pixels can be set asappropriate taking into account factors such as the performance of thecomputer.

The image identification unit 130 includes an identification database131 in which frequency histograms of known products are stored inassociation with the known products. The frequency histogram of knownproducts stored in the identification database 131 may be one thatrepresents a single product or a plurality of types of products. In theinstance of the object classification device 200, as shown in FIG. 2(2),it is possible to provide the image classification means 230 forclassifying by similarity the frequency histograms created by thefrequency histogram creation means 125 instead of the imageidentification unit 130. In the instance of the feature extractiondevice 300, the feature extraction unit 120 being included will suffice;the image identification unit 130 or the image classification means 230are not necessary.

FIG. 3 illustrates the actuation steps regarding the feature extractionunit 120. When a program stored in the program memory 150 is read andexecuted by the control unit 140, first, an image of the subject imagedby the CCD or another means is inputted (S421). Then, the featureextraction unit 120 is actuated in the sequence of: a filter bankcreation step (S422) for creating a filter bank from the image; afiltering result synthesis step (S423) for synthesizing the filteringimage obtained by the filter bank creation step (S422) and creating amaximum brightness image; a maximum brightness image center and radiussetting step (S424) for setting a circular image region of the maximumbrightness image obtained by the filtering result synthesis step (S423)and setting the center Cc and the radius Rc of the circular imageregion; a three-dimensional projection step (S425) for projecting themaximum brightness image in a three-dimensional space; and a frequencyhistogram creation step (S426) for creating a frequency histogram fromthe pixels projected in the three-dimensional projection step (S425).

In the filter bank creation step (S422), multiscale/multidirectionalimage filtering is performed on the image. In the present invention,differences such as creases and folded overlaps originating in thematerial are extracted as a feature description from the imaged image,instead of the shape of the subject being compared with known images.Therefore, inputted image data corresponding to only one image will besufficient.

Performing multiscale/multidirectional image filtering and creating afilter bank are commonly used as a method for classifying textureimages. In this filtering, a variety of changes are made to awave-shaped parameter, and convolution is performed to generate areaction image (reaction to various filters applied to the input image)with respect to each waveform. In the present invention, a Gabor filterin which the phase of the kernel function is displaced by R/2 to obtainan edge detector is used at a variety of amplitudes, window widths,direction components, and the like.

FIG. 4 illustrates specific steps for creating the filter bank. FIG. 4(a) illustrates an imaged image in which a long-sleeved T-shirt placedcasually on the floor represents the subject. The cloth product in theimage can be divided into three types of portions; (1) buttons andsleeves, (2) creased portions, and (3) foldedly overlapped portions andoutline of the cloth. From amongst the above, type (1) representsportions that depend on the type of the cloth product; e.g., sleeves arepresent on a buttoned shirt but not on a towel. In contrast, (2) creasedportions and (3) foldedly overlapped portions of the cloth may bepresent in all cloth products. In the present invention, the state of(2) the creased portions and the state of (3) the foldedly overlappedportions of the cloth, as well as the textile shape and the material ofthe cloth, are extracted as feature descriptions.

FIG. 4(B) illustrates an overview of filtering. First, gentlestripe-shaped edges are detected in the image region in which the clothproduct is imaged. These regions include a high density of frequencieshaving relatively large periods. Therefore, a Gabor filter correspondingto a variety of different amplitudes, window widths, directioncomponents, and the like as described above is applied in order toextract this property from the image data. The individual images in FIG.4(B) represent Gabor filters corresponding to a variety of differentamplitudes, window widths, direction components, and the like. Betweenthe individual images in the horizontal direction in FIG. 4(B), only thedirection of the filter is changed. Thick creases oriented in a specificdirection on the cloth are emphasized according to the direction of thefilter. Between the individual images in the vertical direction in FIG.4(B), the frequency of the filter is gradually made smaller in anincremental manner. As the frequency decreases, thinner creases can beemphasized, and the shape of the textile of the cloth becomes easier toobserve. The frequency of the filter used in the present invention ispreferably varied between 1.0 to 8.0 pixels; in particular, pixel valuesof 1.0 to 3.0 are effective for observing thinner creases and viewingthe difference in textiles. The filter is also used to extract foldedlyoverlapping portions of the cloth. With regards to the black and whitein the image, white portions correspond to portions that exhibited astrong reaction in filtering with regards to gentle creases, and blackportions correspond to portions for which it has been determined thatthinner creases and foldedly overlapped portions ought to be emphasizedin relation to gentle creases (i.e., portions for which it has beendetermined that the portions ought to be used for identification).

The Gabor filter will now be described in further detail. This Gaborfilter is used for scale space construction and the like in wavelettransform, and is represented by the following relationship:

$\begin{matrix}\left\{ {{Numerical}\mspace{14mu}{relationship}\mspace{14mu} 1} \right\} & \; \\{{g\left( {x,\theta,\sigma_{x},\sigma_{y}} \right)} = {\frac{1}{\sqrt{2\pi}\sigma_{x}\sigma_{y}}{\mathbb{e}}^{a}{\cos\left( {{2\pi\;{fx}_{\theta}} + p} \right)}}} & (1)\end{matrix}$

where the following relationship is true:

$\begin{matrix}\left\{ {{Numerical}\mspace{14mu}{relationship}\mspace{14mu} 2} \right\} & \; \\{{a = {{- \frac{1}{2}}\left( {\frac{x_{\theta}^{2}}{\sigma_{x}^{2}} + \frac{y_{\theta}^{2}}{\sigma_{y}^{2}}} \right)}}{x_{\theta} = {{\left( {x - u_{x}} \right)\cos\;\theta} + {\left( {y - u_{y}} \right)\sin\;\theta}}}{y_{\theta} = {{{- \left( {x - u_{x}} \right)}\sin\;\theta} + {\left( {y - u_{y}} \right)\cos\;\theta}}}} & (2)\end{matrix}$

Here, f represents the frequency region. σ² _(x) and σ² _(y) are valuesdetermining the kernel size. As shown in FIG. 5, f is determined inconnection with σ. For example, the value of σ is determined so that thevolume of a shape defined by the horizontal axis of the graph and thecurve representing relationship (1) is uniform, irrespective of theamplitude value. In the present invention, in order to handle x and y ina similar manner, σ_(x) and σ_(y) will be denoted as σ hereafter. Inother words, f(x, θ, σ_(x), σ_(y)) will be referred to as f(x, θ, σ).(x, y) represents the coordinates of the current pixel, and u_(x), u_(y)is the center coordinates for a Gaussian distribution. p is a variablerepresenting the phase; setting π/2 as p constitutes the edge detectorin the present invention.

Since a Gabor filter is directional, it is possible to set θ inrelationship (1) to emphasize an edge in a specific direction. In orderto examine the direction of a crease or the like, filtering is performedwhile varying θ. Specifically, edge detection is performed for each ofeight separate directions between −π≦θ<π. With regards to pixels thatexhibited a negative value during filtering in each direction, the valueis corrected to zero.

Next, in the filtering result synthesis step S(423), the image obtainedby filtering is synthesized, and the maximum brightness image (may bereferred to as “MM image”) is generated. For the MM image, pixel values(brightness values) of pixels that are in the same positions in allimages (images along the horizontal direction in FIG. 4(B)) are addedand a single image is generated (“Gabor total” in FIG. 4(B)). Similarprocesses are performed with a variety of changes being made to σ,whereby a plurality of images are obtained. The MM image is an image inwhich the value of σ at which the maximum brightness value was exhibitedin the images described above is used as the pixel value (brightnessvalue) of each of the pixels.

FIG. 4( c) is the synthesized MM image. Portions having a higherconcentration of darker pixels represent creases, folded overlaps, pile,and other uneven portions, and signify that a strong reaction wasobtained using a filter in which σ has been set to a low value. Brightportions in FIG. 4( c) signify portions of thin creases and portionswhere the cloth is smooth. When this image is represented as I_(mag)(x),the calculation formula is expressed as follows.

{Numerical relationship 3}I _(mag)(x)=argmaxF ₂(x,σ)  (3)

Here,

{Numerical relationship 3}F ₂(x,σ)=∫_(θ)∫_(w) f(x)g(x+x ₀,θ,σ)dx ₀ dθ.  (4)

The window width w and the frequency f are automatically determined fromσ. According to the setup in the present invention, the width of thewindow was defined as w=6×σ, and the frequency was defined as f=1/(3×σ).Meanwhile, the value for the maximum amplitude of the waveform wasobtained using the following relationship.

$\begin{matrix}\left\{ {{Numerical}\mspace{14mu}{relationship}\mspace{14mu} 5} \right\} & \; \\{\lambda_{\max} = {\frac{cons}{x}.}} & (5)\end{matrix}$With the aforementioned settings, the constant “cons” was set to 0.3171.This value was obtained from the result of adjusting the area of theportion between the horizontal axis and the waveform in FIG. 5 so as tobe constant for all values of σ.

Once the maximum brightness image is synthesized, in the maximumbrightness image center and radius setting step (S424), as shown in FIG.6, the clothing region is extracted from the maximum brightness image, acircular region surrounding the region is defined, and the center Cc andthe radius Rc of the region are set.

Once the center Cc and the radius Rc of the maximum brightness imageregion are set, the maximum brightness image is projected in athree-dimensional space in the three-dimensional projection step (S425).The axes for the three-dimensional projection are set as followsaccording to the feature description to be extracted in the maximumbrightness image.

First, a description will be given for the cloth material and wrinkledistribution method (CM-WD method), which is a method for extracting afeature description focusing on the cloth material and the density ofthe wrinkles, in the embodiment of the present invention.

A variety of materials such as cotton and polyester are used to formcloth products, and methods for producing such materials are alsodiverse. In addition to such conditions, the manner in which wrinklesform is also affected by factors such as the thickness of the cloth.Designing a feature description that adequately describes the state ofthe cloth product arising from such differences may make it possible toclassify cloth products. Therefore, as shown in FIG. 7, the followingthree parameters are examined, using the center Cc of the MM imageregion as the origin, with regards to all pixels in the MM image.

1. Ratio between the distance L_(D) between the pixel position (x, y)and the center Cc, and the radius Rc

2. Brightness value F_(D)(x, y) of the pixel

3. Total of the difference between the brightness value F_(D) of thepixel and the brightness value of a nearby pixel

The total of the difference in the brightness values with respect tonearby pixels in item 3 can be represented by the followingrelationship.

{Numerical relationship 6}D _(D)=Σ_(i,jεw)(F _(D)(x,y)−F _(D)(x+i,y+j))  (6)Dv=Li˜jεw(Fv(x,y)−FD(X+i,y+j))  (6)

All of the pixels in the MM image are, as shown in FIG. 8, projected ina three-dimensional space in which the axes represent the aforementionedthree parameters, whereby all of the pixels in the MM image aredistributed in the three-dimensional space.

Next, in the frequency histogram creation step (S426), each of the axesis divided equally into n parts, whereby the three-dimensional space ispartitioned into n³ three-dimensional grids, and the number of pixelspresent in each of the grids is extracted. As shown in FIG. 9, aone-dimensional histogram is obtained with the horizontal axisrepresenting each of the grids and the vertical axis representing thenumber of pixels present in each grid. Feature extraction carried out asdescribed above does not vary according to factors such as the shape,size, and orientation of the cloth; therefore, substantially similarfrequency histograms are obtained for products of the same type. Noparticular limitations exist for the number (n³) of thethree-dimensional grids, although n is preferably 4 to 16. If n is lessthan 4, a single grid will contain an excessively high number of pixels,preventing the feature description from being extracted in anappropriate manner. In contrast, if n is 17 or greater, a single gridwill contain too low a number of pixels, preventing the featuredescription from being extracted in an appropriate manner. Accordingly,it is preferable that n is 8.

Next, a description will be given for the existence of cloth-overlaps(OVLP) method, which is a feature extraction method focusing on theposition and direction of folded overlaps of the cloth and which isanother embodiment of the present invention. In the OVLP method, theaxes projected in the three-dimensional projection step (S425) are setas follows.

The OVLP method is a method that focuses on the filtering results atrelatively short wavelengths, making it possible to extract clothboundaries and folded overlaps of the cloth. FIG. 10 shows an overviewof a description of the OVLP method, based on the output of the MMimage. First, regions of interest such as cloth boundaries and foldedoverlaps of the cloth are set in the MM image, and the following threetypes of parameters are examined with regards to the pixels in theregions of interest (hereafter referred to as “pixels of interest”).

1. Ratio between the distance L_(D) between the position (x, y) of thepixel of interest and the center Cc, and the radius Rc

2. Value E_(O) in which whether the pixel of interest is positioned onthe upper side or the lower side of a folded overlap is evaluated by acontinuous value

3. Direction component of the folded overlap portion in which the pixelof interest is present

Of the above parameters, the “value E_(O) in which whether the pixel ofinterest is positioned on the upper side or the lower side of a foldedoverlap is evaluated by a continuous value” is calculated using thefollowing method. First, an image edge deemed to be a folded overlap(i.e., an edge-part image, corresponding to the snaking thick black lineportion in FIG. 10) is determined from the result in FIG. 4 (Gabortotal), and a straight-line approximation is performed, taking the edgeto be locally linear (dotted line portion in FIG. 10). In thisstraight-line approximation, first, there are readied (1) an image 11 inwhich portions thought to be folded overlaps are white and the remainingportions are black, and (2) an image 12 obtained by grayscaling theoriginal image. Then, the following process is performed, sequentiallyreferencing the white pixels in the image I1. A localized region (in theactual setup, about 16×16 pixels) centered on a given white pixel(referred to here as pixel A) is referenced, and one straight line thatcovers as many white pixels as possible within the localized region isselected. Then, a, b, and c in the equation ax+by+c=0 representing thestraight line are obtained as parameters.

Next, the brightness values of a plurality of pixels on both sides ofthe edge are examined, and whether the pixels are on the upper side orthe lower side of the folded overlap is determined. With regards to thespecific determining procedure, first, with regards to a black pixel inthe vicinity of pixel A (referred to here as pixel B), which side of thestraight line pixel B is present in can be determined by substitutingthe coordinate values of B into x and y in ax+by+c=0. An examination ismade in regard to the brightness values of the pixels in theaforementioned (2) grayscaled image I2 as to which side of the tworegions divided by the straight line contains a greater number of brightbrightness values. The side containing a greater number of pixels havinga bright brightness value as a result is deemed to be the upper side,and the opposite side is deemed to be the lower side, and a map in whicha value of 1 is imparted to pixels in the former side and a value of −1is imparted to pixels in the latter side is temporarily created. Next,with regards to a given pixel and nearby pixels, there is defined avariable for which the values of 1 or −1 imparted thereto are added. Thevariable is then divided by the number of nearby pixels to obtain a realvalue. This real value is the “value E_(O) in which whether the pixel ofinterest is positioned on the upper side or the lower side of a foldedoverlap is evaluated by a continuous value.” This process is performedwith regard to all pixels in the vicinity of the edge.

These results regarding the pixels of interest in the MM image areprojected, as with the CM-WD method, in a three-dimensional space inwhich the axes represent the aforementioned three parameters, and afrequency histogram is obtained. The resulting expression does not varywith respect to scale, rotation, and translation.

(Feature Descriptions for Comparison)

Other than the aforementioned feature extraction methods of the presentinvention (CM-WD method and OVLP method), the following three types offeature descriptions were also extracted and tested as described belowfor comparison. The first is a feature description focusing on creasingin the cloth. The second uses a part of a feature descriptioncalculation commonly used for conventional identification. The thirduses an intermediate processing result obtained in the process ofperforming calculations according to the CM-WD method and OVLP method. Adescription will now be given with regards to each of the featuredescriptions.

(Feature Description Regarding Distribution and Direction of Creases:Distribution of Wrinkle Position (DST-W))

Multidirectional filtering using a filter bank makes it possible toestablish the presence and gradient direction of smooth brightnesschanges such as creases. Regions that are extracted as creases aredivided according to the difference in the direction component, and eachresult of the division is approximated to an ellipse. The number ofdivisions is equal to the number of direction components when the filterbank is generated. A similar approach is adopted in regard to thedivision spacing. The result is projected in a three-dimensional spacein which the axes are the following three parameters.

1. Ratio between the length of the long axis of the ellipse, and theradius Rc

2. Ratio between the length of a line segment L_(E) linking the centerCc and the center coordinates C_(E) of the ellipse region, and theradius Rc

3. Relative angle θ_(E) between the line segment L_(E) and the long axisof the ellipse

The center Cc and the radius Rc are as shown in FIG. 6. FIG. 11illustrates the relationship between the parameters regarding thethree-dimensional projection according to the DST-W method, where C_(E)is the center coordinates of the ellipse region, and θ_(E) is therelative angle between the line segment L_(E) and the long axis of theellipse. This process is performed on all crease regions; then, as withthe CM-WD method, the feature space is partitioned intothree-dimensional grids and the number of values present in each of thegrids is extracted, whereby a frequency histogram is obtained.

(Scale Space Extrema (SSEX))

In a common image feature “SIFT,” extrema in a scale space are used, andfeature description that is robust with respect to scale, translation,and rotation changes is realized. Focusing on the extrema is alsothought to be useful in the present invention for describing the clothmaterial and the manner in which creases are formed. The followingfeature descriptions are used from the detected extrema and thesurroundings thereof.

1. Scale value of the extremus

2. Ratio between the radius Rc and the distance between the center Ccand the respective extremus

3. Ratio between the radius Rc and the distance between the presentextremus and the nearest extremus

The center Cc and the radius Rc are as shown in FIG. 6. As with theaforementioned CM-WD method, projection to a three-dimensional space isperformed, the feature space is partitioned into three-dimensionalgrids, and the number of values present in each grid is extracted,whereby a frequency histogram is obtained.

(Outline of Cloth Region: Contour (CNTR))

It is necessary to extract the region of the cloth product aspre-processing for the aforementioned extraction of the featuredescription (CM-WD method and OVLP method) according to the embodimentof the present invention. Specifically, the outline of the cloth productis obtained in the course of the feature description calculation.Accordingly, a feature description is also calculated according to thefollowing step from the outline. First, the image is binarized to thecloth region and other regions. The image is then subjected to a polarcoordinate conversion, and a θ-r graph centered on the center Cc isgenerated. Here, θ represents the tilt angle of the straight linepassing through the center Cc when the horizontal axis of the imagerepresents zero degrees, and r is the distance on the line segment fromthe center Cc to the outline. It is thereby possible to obtain adistance histogram relating to the boundary line. The histogram is thendiscretized at arbitrary spacings, and a feature vector is generated.

Next, an example of a step for creating the identification database 131included in the image identification unit 130 will now be described withreference to FIG. 12. First, a plurality of known products are readiedas subjects, and a plurality of images captured from a variety of anglesare readied for each of the known products. Then, the first image of thefirst known product is inputted (S531), a frequency histogram is created(S532) according to the aforementioned steps (S422)-(S426), it isdetermined whether or not there are any unprocessed images among theplurality of images of a single known product prepared in advance(S533), and if there are any unprocessed images, the steps (S531)-(S532)are repeated. If there are no unprocessed images, creation of thefrequency histogram of the first known product is completed, and it isdetermined whether or not there is an image of the next known product(S534). If there is an image of the next known product, the steps(S531)-(S533) are repeated. Once processing of the readied images of theknown product is complete (“No” in S534), the flow proceeds to a step inwhich the created frequency histogram is associated with the knownproduct and stored in the database (S535). In this step, the frequencyhistograms are grouped for each known product for which the frequencyhistogram has been created, a boundary surface is set at the boundarywith respect to a frequency histogram group for another known product,and each group is associated with a known product and stored in thedatabase, whereby the identification database 131 is created. Theboundary surface can be set using a known machine learning method suchas support vector machine (SVM). The number of images prepared inadvance for each known product varies according to the desiredidentification accuracy, and no particular lower limit therefore exists.However, since the identification error decreases with more images, thenumber of images to be prepared is preferably 200 or more.

Other than the aforementioned method, the identification database 131may also be created, e.g., by associating individual frequencyhistograms created from a plurality of images of a known product andstoring the frequency histograms in a database. No particularlimitations exist, as long as the method is a well-known one used tosolve classification problems in machine learning and similar fields.

Then, in the steps shown in FIG. 3, a frequency histogram is createdfrom the image of the subject to be identified, and it is determinedwhich of the frequency histogram groups stored in the identificationdatabase 131 the frequency histogram belongs to, whereby it is possibleto identify the subject. If the object identification device of thepresent invention is applied, e.g., to a transportation line of acleaning plant, the cleaned item, such as a buttoned shirt, ahandkerchief, or a towel which has been transported is imaged as asubject, a frequency histogram is created from the imaged image, thesubject is then identified using the identification database 131, and{the subject} is then automatically transported to the processing stepcorresponding to the identification result, whereby sorting of cleaneditems and transportation of the cleaned items to the processing step ina cleaning plant can be automated.

If the subjects are classified into similar groups without identifyingthe subjects, there is no need to provide an identification database131; instead, it is possible to provide image classification means 230for classifying similar frequency histograms into the same groups andconstitute an object classification device. For the image classificationmeans 230, known image classification means such as repeated bisection(non-patent document 7) or K-means can be used. If the objectidentification device of the present invention is applied, e.g., to atransportation line of a cleaning plant, it is possible to image abuttoned shirt, a handkerchief, a towel, or the like as a subject,create a frequency histogram from the imaged image, classify similarfrequency histograms into the same groups, and thereby automaticallysort cleaned items.

{Embodiments}

(Products Used in Embodiment)

9 classes of everyday-use cloth products (1) to (9) shown in the uppersection of FIG. 13 were used in the embodiment. The left side of thelower section shows the cloth product (7) and the right side of thelower section shows the cloth product (9) in a state of being foldedlyoverlapped as seen in real life. The type and material of the clothproduct are as follows: (1) turtleneck sweater (100% cotton); (2) thickparka (100% polyester); (3) sweater (100% cotton); (4) cardigan (100%acrylic); (5) dress (100% cotton); (6) T-shirt (100% cotton); (7) handtowel (100% cotton); (8) hand towel (100% cotton); and (9) shirt (78%cotton, 22% polyester). Both (7) and (8) are pile material towels, buthaving different sizes and shapes when spread.

(Classification of Subject)

Unsupervised clustering was performed to evaluate whether or not thefrequency histogram obtained by the feature extraction method of thepresent invention is suitable for classification of subjects. Repeatedbisection was used as the clustering method. This method is one forautomatically grouping similar frequency histograms without an externalstandard in which the provided data is stored in a database or the like.If the frequency histograms extracted using the aforementionedextraction method are similar, the frequency histograms are classifiedby the type of the cloth product, irrespective of the manner in whichthe cloth product is placed.

In the present embodiment, the number n of divisions in relation to eachaxis of a three-dimensional space when a frequency histogram is createdwas set to eight. Unsupervised clustering was performed on approximately250 frequency histograms on each of cloth products (6) to (9), i.e., atotal of approximately 1000 frequency histograms. The number of clusters(number of groups to which classification is performed) was set to 10.Table 1 shows results regarding the CM-WD method, which is theembodiment of the present invention, and table 2 shows results regardingthe SSEX method, which is a comparative example.

TABLE 1 Cluster 1 2 3 4 5 6 7 8 9 10 (6) Tshirt 0 64 5 0 0 0 0 47 112 1(7) Towel 58 0 2 3 82 85 12 47 1 9 (8) Towel 2 0 0 0 28 24 95 1 11 72(9) Shirt 0 1 119 124 0 0 0 2 0 0

TABLE 2 Cluster 1 2 3 4 5 6 7 8 9 10 (6) Tshirt 40 68 1 71 5 28 1 30 0 2(7) Towel 3 10 34 2 23 11 102 28 53 33 (8) Towel 10 3 12 2 23 6 14 52 1464 (9) Shirt 3 30 41 15 10 92 16 5 16 1

As can be seen in table 2, according to the conventional SSEX method,even though grouping into cloth products is performed to a certainextent, a variety of cloth products are classified into groups(clusters) in which the feature description is similar (e.g., 68T-shirts (6), 30 shirts (9), 10 towels (7), and 3 towels (8) areclassified in cluster 2) and no correspondence relationship can beidentified between feature descriptions and cloth products.

In contrast, with regards to the CM-WD method, which is the featureextraction method of the present invention in table 1, it can beconfirmed that identical products will be classified into the samecluster irrespective of the shape of the cloth product, in contrast tothe conventional SSEX method (e.g., cluster 2 containing 64 T-shirts(6), only one shirt (9), and zero towels (7) or towels (8)), and acorrespondence relationship can be identified between the featuredescriptions and the cloth products. Therefore, using the featureextraction method according to the present invention makes it possibleto classify similar subjects into identical groups irrespective of theshape such as that arising from creases and folded overlaps and withoutusing identification means for identifying the subject.

(Identification of Subject)

Identification is performed using machine learning methodology todetermine whether or not a frequency histogram obtained using thefeature extraction method according to the present invention is suitablefor identification of subjects. First, in order to identify the clothproduct to which the frequency histogram extracted from the image of thesubject corresponds, a cloth product was casually placed on a flatsurface such as a floor or a table for each of the cloth products (1) to(9) shown in FIG. 13, and approximately 250 images were captured foreach of the cloth products. An image size corresponding to VGA (640×480pixels) was used, and approximately 2400 items of image data wereobtained. With regards to each of the images, in the steps shown in FIG.12, a frequency histogram according to the CM-WD method and the OVLPmethod representing the present invention and a frequency histogramaccording to the DST-W method and the SSEX method were created, thefrequency histograms were grouped according to product, the boundarysurface between each of the groups is set and the frequency histogramswere stored in a database in association with known products, whereby anidentification database 131 was created. Multiclass Support VectorMachine (SVM) was used in the creation of the identification database131. For the setup, LIBSVM (non-patent document 8) was used, and C-SVCand RBF kernel were set for the identifying means and the kernelfunction, respectively. With regards to the CNTR method, as describedabove, distance histograms relating to the boundary line werediscretized at arbitrary spacings, and the generated feature vector wasassociated with a known product and stored in the database.

N-fold cross-validation was applied as the method for evaluating theperformance of the identification database 131. FIG. 14 shows thevariations in identification rates as N is varied from 2 to 10 for theCM-WD and OVLP methods of the present invention and the DST-W, SSEX, andCNTR methods of the comparative examples. Again, the CM-WD method, whichis a feature extraction method of the present invention, exhibited thebest result with an identification rate of 89%, while the identificationrate of the SSEX method, which is a comparative example, was merely 61%.

Next, with regards to the CM-WD, OVLP, SSEX, and DST-W methods, two orthree types of the corresponding histograms were linked, and anidentification database 131 was created as learning data. Linkinghistograms refer to arranging one type of histogram in succession afteranother type of histogram. For example, in FIG. 15, “CM-WD OVLP” refersto the histogram for the OVLP method being arranged in succession afterthe histogram for the CM-WD method. As can be seen from FIG. 15, whenCM-WD and SSEX are arranged next to each other, the identification ratereaches 98% at its highest, revealing that the type of the cloth productcan be correctly identified in almost all instances irrespective of theshape of the cloth product.

In the graph, the identification rate becomes stable where the value ofN is 4 to 5. It is thereby revealed that about 200 is sufficient for thenumber of items of learning data, even though cloth products can take onan infinite number of shapes.

Industrial Applicability

Through installation on a robot or another machine, it is possible toextend the range of items that can be handled by the present inventionfrom soft items to rigid bodies, and the present invention is useful inthe fields of IT, automotive vehicles, production machinery, andmedicine/healthcare. In particular, in the present invention, thefeature description can be extracted irrespective of the outer shape ofthe subject without using a special physical means, and the type of thesubject can be identified according to the feature description.Therefore, the present invention is useful for: automation of handlingof soft items such as automatic sorting of objects being cleaned in acleaning plant; automation of identifying the state of use of a productsuch as a shirt, through extracting, as a feature description, thetemporal change due to use of the product; automation of identifyingclothing worn by pedestrians through cameras installed on the street;development of automatic travel systems for electric wheelchairs andautomobiles; development of automatic vacuum cleaners that focuscleaning on soiled portions where dust, refuse, or stains are present;identification of similarities between cities through extraction offeature descriptions from aerial photographs; automation ofclassification of vegetables, fruits, or the like; automation ofidentifying freshness of vegetables, fruits, and the like throughextracting changes in the feature description due to temporal change invegetables, fruits, or the like; and improvement to the learningfunction in libraries, museums, and resource centers and the likethrough sending images of subjects such as birds and plants online andidentifying the subject type.

The invention claimed is:
 1. A feature extraction method for extractinga feature description from an image of a subject, the feature extractionmethod comprising the steps of: capturing the image with an imagecapturing device; electrically creating a filter bank from said image;electrically creating a maximum brightness image from said filter bank;electrically setting a circular image region of said maximum brightnessimage, and setting a center Cc and a radius Rc of the circular imageregion; electrically projecting a pixel in the maximum brightness imagein a three-dimensional space having axes representing (a) a ratiobetween a distance L_(D) between a pixel position (x, y) of the pixeland said center Cc, and said radius Rc, (b) a brightness value F_(D)(x,y) of the pixel, and (c) the total of the difference between thebrightness value F_(D) of the pixel and the brightness value of a nearbypixel; and electrically creating a frequency histogram from the pixelsprojected in said three-dimensional space.
 2. The feature extractionmethod according to claim 1, said subject being a soft item.
 3. Thefeature extraction method according to claim 1, said filter bank beingcreated using a Gabor filter.
 4. An object classification methodcomprising: electrically classifying the subject using the frequencyhistogram extracted using the feature extraction method according toclaim
 1. 5. An object identification method comprising; electricallycomparing the frequency histogram extracted using the feature extractionmethod according to claim 1 to a frequency histogram of a known subject;and identifying a type of the subject.
 6. The object identificationmethod according to claim 5, wherein the comparing the extractedfrequency histogram with a frequency histogram of a plurality of theknown subject types, whereby the type of the subject is identified.
 7. Acomputer program stored in a non-transitory computer readable medium,the computer program configuring a feature extraction device to executethe feature extraction method according to claim
 1. 8. A non-transitorycomputer-readable medium that, when executed with a feature extractiondevice, causes the feature extraction device to execute the featureextraction method according to claim
 1. 9. A feature extraction methodfor extracting a feature description from an image of a subject, thefeature extraction method comprising: capturing the image with an imagecapturing device; electrically creating a filter bank from said image;electrically creating a maximum brightness image from said filter bank;electrically setting a circular image region of said maximum brightnessimage, and setting a center Cc and a radius Rc of the circular imageregion; electrically projecting a pixel in the maximum brightness imagein a three-dimensional space having axes representing (d) a ratiobetween the distance L_(o) between the position (x, y) of the pixel andsaid center Cc, and said radius Rc, (e) a value E_(o) in which whetherthe pixel is positioned on an upper side or a lower side of a foldedoverlap is evaluated by a continuous value, and (f) a directioncomponent of the folded overlap portion in which the pixel is present;and electrically creating a frequency histogram from the pixel projectedin said three -dimensional space.
 10. A feature extraction device forextracting a feature description from an image of a subject captured byimage-capturing means, the feature extraction device comprising: filterbank creation means for creating a filter bank from said image;filtering result synthesizing means for creating a maximum brightnessimage from said filter bank; maximum brightness image center and radiussetting means for setting a circular image region of said maximumbrightness image, and setting a center Cc and a radius Rc of thecircular image region; three-dimensional projection means for projectinga pixel in the maximum brightness image in a three-dimensional spacehaving axes representing (a) a ratio between the distance L_(D) betweenthe pixel position (x, y) of the pixel and said center Cc, and saidradius Rc, (b) the brightness value F_(D)(x, y) of the pixel, and (c)the total of the difference between the brightness value F_(D) of thepixel and the brightness value of a nearby pixel; and frequencyhistogram creation means for creating a frequency histogram from thepixels projected in said three-dimensional space.
 11. The featureextraction device according to claim 10, wherein said subject being asoft item.
 12. The feature extraction device according to claim 10,wherein said filter bank being created using a Gabor filter.
 13. Anobject classification device comprising image classification means forclassifying a subject using the frequency histogram extracted using thefeature extraction device according to claim
 10. 14. An objectidentification device comprising: an identification database storing afrequency histogram of a known subject, and identification means forcomparing, with the frequency histogram of the known subject stored insaid identification database, and identifying, the frequency histogramextracted by the feature extraction device according to claim
 10. 15.The object identification device according to claim 14, wherein: thefrequency histograms of the known subject stored in said identificationdatabase are frequency histograms of a plurality of types of subjects,and said identification means compares said extracted frequencyhistogram with the plurality of types of frequency histograms stored insaid identification database, and thereby identifies the type of thesubject.